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Abstract. Probabilistic timed automata are an extension of timed automata with dis- 
crete probability distributions. We consider model-checking algorithms for the subclasses 
of probabilistic timed automata which have one or two clocks. Firstly, we show that 
PCTL probabilistic model-checking problems (such as determining whether a set of tar- 
get states can be reached with probability at least 0.99 regardless of how nondetermin- 
ism is resolved) are PTIME-complete for one-clock probabilistic timed automata, and are 
EXPTIME-complete for probabilistic timed automata with two clocks. Secondly, we show 
that, for one-clock probabilistic timed automata, the model-checking problem for the prob- 
abilistic timed temporal logic Ptctl is EXPTIME-complete. However, the model-checking 
problem for the subclass of Ptctl which does not permit both punctual timing bounds, 
which require the occurrence of an event at an exact time point, and comparisons with 
probability bounds other than or 1, is PTIME-complete for one-clock probabilistic timed 
automata. 



Model checking is an automatic method for guaranteeing that a mathematical model 
of a system satisfies a formally-described property |CGP99| . Many real-life systems, such 
as multimedia equipment, communication protocols, networks and fault-tolerant systems, 
exhibit probabilistic behaviour. This leads to the study of model checking of probabilistic 
models based on Markov chains or Markov decision processes |Var851 IHJ94t ICY951 IBdA951 
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Table 1: Complexity results for model checking probabilistic timed automata 





One clock 


Two clocks 


Reachability, Pctl 


P-complete 


EXPTIME-complete 


Ptctl°/1[<,>] 
Ptctl^/-"^ 


P-complete 
EXPTIME-complete 


EXPTIME-complete 
EXPTIME-complete 


Ptctl[<, >] 
Ptctl 


P-hard, in EXPTIME 
EXPTIME-complete 


EXPTIME-complete 
EXPTIME-complete 



ldA97al lBK98j . Similarly, it is common to observe complex real-time behaviour in systems. 
Model checking of (non-probabilistic) continuous-time systems against properties of timed 
temporal logics, which can refer to the time elapsed along system behaviours, has been 
studied extensively in, for example, the context of timed automata [ACD93t IAD94| . which 
are automata extended with clocks that progress synchronously with time. Finally, certain 
systems exhibit both probabilistic and timed behaviour, leading to the development of 
mode l -checking algorithms for such systems |A(]D9H ULTMI ldA97al IKNSSn2[ IBHHKO.Sl 
ILSM mi6| lBCH+n7l IDHSn7j . 

In this paper, we aim to study model-checking algorithms for probabilistic timed au- 
tomata [ Jen96t IKNSS02] . which can be regarded as a variant of timed automata extended 
with discrete probability distributions, or (equivalently) Markov decision processes extended 
with clocks. Probabilistic timed automata have been used to model systems such as the 
IEEE 1394 root contention protocol, the backoff procedure in the IEEE 802.11 Wireless 
LANs, and the IPv4 link local address resolution protocol |KNPS06] . The temporal logic 
that we use to describe properties of probabilistic timed automata is Ptctl (Probabilistic 
Timed Computation Tree Logic) |KNSS02] . The logic Ptctl includes operators that can 
refer to bounds on exact time and on the probability of the occurrence of events. For exam- 
ple, the property "a request is followed by a response within 5 time units with probability 
0.99 or greater" can be expressed by the Ptctl property request P>o.99(F<5response). 
The logic Ptctl extends the probabilistic temporal logic Pctl [HJ941 IBdA95j . and the 
real-time temporal logic Tctl [ACD93] . 

In the non-probabilistic setting, timed automata with one clock have recently been 
studied extensively |LMS04t ILWOSj lADOWOS] . In this paper we consider the subclasses of 
probabilistic timed automata with one or two clocks. While probabilistic timed automata 
with a restricted number of clocks are less expressive than their counterparts with an arbi- 
trary number of clocks, they can be used to model systems with simple timing constraints, 
such as probabilistic systems in which the time of a transition depends only on the time 
elapsed since the last transition. Conversely, one-clock probabilistic timed automata are 
more natural and expressive than Markov decision processes in which durations are asso- 
ciated with transitions (for example, in [dA97bl ILS05| ). We note that the IEEE 802.11 
Wireless LAN case study has two clocks |KNPS06] . and that an abstract model of the 
IEEE 1394 root contention protocol can be obtained with one clock [Sto02 j . 

After introducing probabilistic timed automata and Ptctl in Section [2] and Section [3l 
respectively, in Section |4] we show that model-checking properties of Pctl, such as the 
property F>Q,Qg{f target) ("a set of target states is reached with probability at least 0.99 
regardless of how nondeterminism is resolved"), is PTIME-complete for one clock prob- 
abilistic timed automata, which is the same complexity as for probabilistic reachability 
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properties on (untimed) Markov decision processes |PT87] . We also show that, in gen- 
eral, model checking of Ptctl on one clock probabilistic timed automata is EXPTIME- 
complete. However, inspired by the efficient algorithms obtained for non-probabilistic one 
clock timed automata [LMS04j . we also show that, restricting the syntax of Ptctl to the 
sub-logic in which (1) punctual timing bounds and (2) comparisons with probability bounds 
other than or 1, are disallowed, results in a PTIME-complete model-checking problem. 
In Section [5l we show that reachability properties with probability bounds of or 1 are 
EXPTIME-complete for probabilistic timed automata with two or more clocks, implying 
EXPTIME-completeness of all the model-checking problems that we consider for this class 
of models. Our complexity results are summarized in Table [H where 0/1 denotes the sub- 
logics of Ptctl with probability bounds of and 1 only, and [<, >] denotes the sub-logics of 
Ptctl in which punctual timing bounds are disallowed. The EXPTIME-hardness results 
are based on the concept of countdown games, which are two-player games operating in 
discrete time in which one player wins if it is able to make a state transition after exactly 
c time units have elapsed, regardless of the strategy of the other player. We show that the 
problem of deciding the winning player in countdown games is EXPTIME-complete. We 
believe that countdown games are of independent interest, and note that they have been 
used to show EXPTIME-hardness of model checking punctual timing properties of timed 
concurrent game structures [LMO06| . Finally, in Section [6l we consider the application of 
the forward reachability algorithm of Kwiatkowska et al. |KNSS02] to one-clock probabilis- 
tic timed automata, and show that the algorithm computes the exact probability of reaching 
a certain state set. This result is in contrast to the case of probabilistic timed automata 
with an arbitrary number of clocks, for which the application of the forward reachability 
algorithm results in an upper bound on the maximal probability of reaching a state set, 
rather than in the exact maximal probability. Note that, throughout the paper, we restrict 
our attention to probabilistic timed automata in which positive durations elapse in all loops 
of the system. 

2. Probabilistic Timed Automata 

2.1. Preliminaries. We use M>o to denote the set of non-negative real numbers, Q to 
denote the set of rational numbers, N to denote the set of natural numbers, and AP to 
denote a set of atomic propositions. A (discrete) probability distribution over a countable 
set Q is a function n : Q ^ [0,1] such that J2qeQ /^(^) ~ ^ function ^ : Q ^ M>o we 

define support(/x) = {q & Q \ fJ-iq) > 0}. Then for an uncountable set Q we define Dist(Q) 
to be the set of functions fi : Q ^ [0)1]) such that support(/i) is a countable set and fj, 
restricted to support(//) is a (discrete) probability distribution. In this paper, we make the 
additional assumption that distributions assign rational probabilities only; that is, for each 
fi G Dist(Q) and q € Q,we have n{q) G [0, 1] fl Q. 

We now introduce timed Markov decision processes, which are Markov decision processes 
in which rewards associated with transitions are interpreted as time durations. 

Definition 2.1. A timed Markov decision process (TMDP) T = {S,s, — > ,lab) comprises 
the following components: 

• A (possibly uncountable) set of states S with an initial state s G 5. 
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A (possibly uncountable) timed probabilistic, nondeterministic transition relation 
C S X M>o X Dist(S') such that, for each state s E S, there exists at least one tuple 



• A labelling function lab : S 2^^ . 



The transitions from state to state of a TMDP are performed in two steps: given that the 
current state is s, the first step concerns a nondeterministic selection of (s, d, z^) € —^, where 
d corresponds to the duration of the transition; the second step comprises a probabilistic 

choice, made according to the distribution i/, as to which state to make the transition to 
(that is, we make a transition to a state s' G S* with probability l'(s')). We often denote 

such a completed transition by s s'. 

An infinite path of the TMDP T is an infinite sequence of transitions oj = sq 
si ^''^^ • • • such that the target state of one transition is the source state of the next. 
Similarly, a finite path of T is a finite sequence of consecutive transitions u = sq ^°'''°) 
duin^ n-i.t^n-i^ ^j^^ length of w, denoted by |a;|, is n (the number of transitions 
along uj). Wc use Pathf^i to denote the set of infinite paths of T, and Path fin the set of 
finite paths of T. If w is a finite path, we denote by last{u) the last state of oj. For any path 
UJ and i < let u;(i) = Sj be the (i + l)th state along uj. Let Pathfui{s) and Pathfin{s) 
refer to the sets of infinite and finite paths, respectively, commencing in state s € S*. 

In contrast to a path, which corresponds to a resolution of nondeterministic and prob- 
abilistic choice, an adversary represents a resolution of nondeterminism only. Formally, an 
adversary of a TMDP T is a function A mapping every finite path u> G Path fin to a transition 
{last{uj),d, I/) G — > . Let Advj be the set of adversaries of T (when the context is clear, we 
write simply Adv). For any adversary A G Adv, let Pathf^i and Path fin denote the sets of 
infinite and finite paths, respectively, resulting from the choices of distributions of A, and, 
for a state s E S, let Pathf^i{s) = Pathf^i PI Pathfui{s) and Pathfin{s) = Pathfi^ fl Pathfin{s). 
Note that, by defining adversaries as functions from finite paths, wc permit adversaries 
to be dependent on the history of the system. Hence, the choice made by an adversary 
at a certain point in system execution can depend on the sequence of states visited, the 
nondeterministic choices taken, and the time elapsed from each state, up to that point. 

Given an adversary A G Adv and a state s G S", we define the probability measure 
Probf over Pathf^i{s) in the following way. We first define the function A : Pathfin{s) x 
Pathfin{s) [0, 1]. For two finite paths u}fin,u)fi^ G Pathfinis), let: 

ACw* uj'^ )- I ^('^') ^'nn is of the form ujfin ^ s' and A{ujfin) = {d, n) 

"^y-" fin I '■"fin) \ „ ii • 

I (J otherwise. 

Next, for any finite path ujfin G Pathfin{s) such that \ujfin\ = n, we define the probability 
P^{ujfin) as follows: 

T}A( N d|f / 1 if n = 

. y^fin) y A{u;fin{0),u;fin{l)) ■ . . . • A{u;fin{n-l),u;fin{n)) otherwise. 

Then we define the cylinder of a finite path ujfin as: 

cyl^{ufin) = {oJ G Pathf^i{s) \ Ufin is a prefix of , 
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and let be the smallest sigma-algebra on Path^i(s) which contains the cylinders cyl^{ijjfin) 
for ojfin G Path^^{s). Finally, we define Probf on as the unique measure such that 
Probf{cyl{ujfir,)) = Pfiujfin) for ah a;^„ G Pathj^{s). 

An untimed Markov decision process (MDP) {S, s, , lab) is defined as a finite-state 
TMDP, but for which — > C 5* x Dist(5') (that is, the transition relation — s- does not 
contain timing information). Paths, adversaries and probability measures can be defined 
for untimed MDPs in the standard way (see, for example, |BK98| ). 

In the remainder of the paper, we distinguish between the following classes of TMDP. 

• Discrete TMDPs are TMDPs in which (1) the state space 5 is finite, and (2) the transition 
relation — > is finite and of the form C 5 x N x Dist(5). In discrete TMDPs, the delays 
are interpreted as discrete jumps, with no notion of a continuously changing state as time 
elapses. The size |T| of a discrete TMDP T is 15"! + | ^ |, where | — > | includes the size of 
the encoding of the timing constants and probabilities used in — > : the timing constants 
are written in binary, and, for any s,s' £ S and {s, d, u), the probability i^{s') is expressed 
as a ratio between two natural numbers, each written in binary. We let be the untimed 
Markov decision process (MDP) corresponding to the discrete TMDP T, in which each 
transition (s, d, i^) G — > is represented by a transition {s,u). A discrete TMDP T is 

structurally non-Zeno when any finite path of T of the form sq ^°''^"> si • • • °'"''^"> Sn+i, 
such that Sn+i = sq, satisfies J2o<i<n'^i > ^^ 

• Continuous TMDPs are infinite-state TMDPs in which any transition s s' describes 

the continuous passage of time, and thus a path uj = sq '^"''^°> si ^^''^^ • • • describes 
implicitly an infinite set of visited states. In the sequel, we use continuous TMDPs to 
give the semantics of probabilistic timed automata. 

2.2. Syntax of probabilistic timed automata. Let Af be a finite set of real-valued 
variables called clocks, the values of which increase at the same rate as real-time. The set 
CC (X) of clock constraints over X is defined as the set of conjunctions over atomic formulae 
of the form x ~ c, where x,y € X , ^ G {<, <, >, >}, and c G N. 

Definition 2.2. A probabilistic timed automaton (PTA) P = (L, /, X, inv,prob,C) is a tuple 
consisting of the following components: 

• A finite set L of locations with the initial location I £ L. 

• A finite set X of clocks. 

• A function inv : L — > CC{X) associating an invariant condition with each location. 

• A finite set prob C L x CC{X) x Dist(2'^ x L) of probabilistic edges. 

• A labelling function C : L ^ 2^^ . 

A probabilistic edge il,g,p) G prob is a triple containing (1) a source location I, (2) 
a clock constraint g, called a guard, and (3) a probability distribution p which assigns 
probabilities to pairs of the form {X, I') for some clock set X X and target location /'. 
The behaviour of a probabilistic timed automaton takes a similar form to that of a timed 
automaton |AD94| : in any location time can advance as long as the invariant holds, and a 
probabilistic edge can be taken if its guard is satisfied by the current values of the clocks. 
However, probabilistic timed automata generalize timed automata in the sense that, once a 
probabilistic edge is nondeterministically selected, then the choice of which clocks to reset 
and which target location to make the transition to is probabilistic. We require that the 
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values of the clocks after taking a probabilistic edge satisfy the invariant conditions of the 
target locations. 




Figure 1: A probabilistic timed automaton P 



Example 2.3. A PTA P is illustrated in Figure [TJ The PTA represents a simple communi- 
cation protocol, in which the sender can wait for between 5 and 6 time units before sending 
the message, at which point the message is delivered successfully with probability 0.8, or 
can wait for between 7 and 8 time units before sending the message, which corresponds to 
the message being sent successfully with probability 0.9. From location wait, there are two 
probabilistic edges: the upper one has the guard 5 < x < 6, and assigns probability 0.8 to 
({x}, init) and 0.2 to (0, error), whereas the lower one has the guard 7 < x < 8, and assigns 
probability 0.9 to ({x}, init) and 0.1 to (0, error). 

The size |P| of the PTA P is \L\ + l-^"! + \inv\ + |pro6|, where \inv\ represents the size 
of the binary encoding of the constants used in the invariant condition, and |7?ro6| includes 
the size of the binary encoding of the constants used in guards and the probabilities used in 
probabilistic edges. As in the case of TMDPs, probabilities are expressed as a ratio between 
two natural numbers, each written in binary. 

In the sequel, we assume that at least 1 time unit elapses in all structural loops within 
a PTA. Formally, a PTA is structurally non-Zeno [TYB05| if, for every sequence Xq, 
ilo,go,Po),Xi,{li,gi,pi), - ■ ■ ,Xn,{ln,gn,Pn), such that pi{Xi+i,li+i) > for < z < n, 
and pn(Xo,/o) > 0, there exists a clock x £ A' and < i,j < n such that x G and 
gj ^ X > 1 (that is, gj contains a conjunct of the form x > c for some c > 1). 

We also assume that there are no deadlock states in a PTA. This can be guaranteed 
by assuming that, in any state of a PTA, it is always possible to take a probabilistic 
edge, possibly after letting time elapse, a sufficient syntactic condition for which has been 
presented in |Spr01| . First, for a set X X clocks, and clock constraint ip G CC{X), 
let [X := Ojtp be the clock constraint obtained from -0 by letting, for each x £ X, each 
conjunct of the form x > c or x > c' where c' > 1 be equal to false. For a clock 
constraint ip G CC{X), let upper('0) be the clock constraint obtained from ifj by substituting 
constraints of the form x < c with x>c— lAx<c, and constraints of the form x < c 
with X > c A X < c. Then, for an invariant condition inv{l) of a PTA location, the 
clock constraint upper(in?;(/)) represents the set of clock valuations for which a guard of a 
probabilistic edge must be enabled, otherwise the clock valuations correspond to deadlock 



MODEL CHECKING PROBABILISTIC TIMED AUTOMATA 



7 



states from which it is not possible to let time pass and then take a probabilistic edge. Then 
a PTA has non- deadlocking invariants if, for each location Z € L, we have upper(mw(/)) =^ 
V(i,g,p)eprofe(ff/\A(x,i')esupport(p) ■= 0]inv {l')). The condition of non-deadlocking invariants 
usually holds for PTA models in practice |KNPS06] . 

We use IC-PTA (respectively, 2C-PTA) to denote the set of structurally non-Zeno PTA 
with non-deadlocking invariants, and with only one (respectively, two) clock(s). 

2.3. Semantics of probabilistic timed automata. We refer to a mapping v : X ^ M>o 
as a clock valuation. Let M>g denote the set of clock valuations. Let G ]R>q be the clock 
valuation which assigns to all clocks in X. For a clock valuation v G R>q and a value 
d £ M>o, we use v + dto denote the clock valuation obtained by letting {v + ct)(x) = v{x) + d 
for all clocks x & X. For a clock set X <Z X, we let v[X := 0] be the clock valuation obtained 
from V by resetting all clocks within X to 0; formally, we let v[X := 0](x) = for all x G X, 
and let v[X := 0]{x) = v{x) for all x G \ X. The clock valuation v satisfies the clock 
constraint G CC{X), written v \= ijj, li and only if ip resolves to true after substituting 
each clock x € X with the corresponding clock value v{x). 

We now present formally the semantics of PTA in terms of continuous TMDPs. The 
semantics has a similar form to that of non-probabilistic timed automata |AD94| . but with 
the addition of rules for the definition of a timed, probabilistic transition relation from the 
probabilistic edges of the PTA. 

Definition 2.4. The semantics of the probabilistic timed automaton P = {L,l,X,inv, 
prob,C) is the continuous TMDP T[P] = {S,s, — > , lab) where: 

• S = {{l,v) I / G L and v G M>q s.t. v \= inv{l)} and s = {1,0); 

• ^ is the smallest set such that {{l,v),d, fi) G ^ if there exist d G M>o and a 
probabilistic edge {l,g,p) G prob such that: 

(1) v + d\= g, and v + d' \= inv{l) for ah < d' < d; 

(2) for any (X, /') G 2^^ x L, we have that p{X, I') > implies {v + d)[X := 0] \= inv{l'); 

(3) for any {l',v') G S, we have that n{l',v') = Y.xeReset{v,d,v')Pi^^ni where 

Reset(7;, d, v') = {X C X \ {v + d)[X := 0] = v'} 

• lab is such that lab{l,v) = C{1) for each state {l,v) G S. 

Given a path uj = (/q, fo) '^°''^°> {hiVi) — Li-U • • • of T[P], for every i G N, we use oj{i, d), 
with < d < di, to denote the state {li,Vi + d) reached from {li,Vi) after delaying d time 
units. Such a pair {i,d) is called a position of w. We define a total order on positions 
of io: given two positions {i, d), {j, d') of to, the position {i,d) precedes (j, d') — denoted 
(i, d) {j, d') — if and only if either i < j, or i = j and d < d' . 

3. Probabilistic timed temporal logic 

We now proceed to describe a probabilistic, timed temporal logic which can be used to 
specify properties of probabilistic timed automata |KNSS02j . 

Definition 3.1. The formulae of Ptctl (Probabilistic Timed Computation Tree Logic) 
are given by the following grammar: 

$ ::= a I $ A $ 1 [ Pt>4^(«>U^c^) 
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where a € AP is an atomic proposition, ixiG {<,<,>,>}, ~€ {<,=,>}, C G [Oi 1] is a 
probability, and c G N is a natural number. 

We use standard abbreviations such as true, false, <I>i V<I>2, =^ ^2, and P|xi^(F^c<I>) 
(for Pt><|^(trueU^c^))- Formulae with "always" temporal operators G^c can also be written; 
for example P>(;(G^c^) can be expressed by P<i_^(F^c~'*^)- The modalities U, F and G 
without subscripts abbreviate U>o, F>o and G>o, respectively. 

We identify the following sub-logics of Ptctl. 

• Ptctl [<, >] is defined as the sub- logic of Ptctl in which subscripts of the form = c are 
not allowed in modalities U^c, F^c, G^^c- 

• PCTL is defined as the sub-logic of Ptctl (and Ptctl [<, >]) in which there is no timing 
subscript ~ c associated with the modalities U, F, G. 

• Ptctl°/^ and Ptctl°/^[<, >] are the sub- logics of Ptctl and Ptctl [<, >], respectively, 
in which probability thresholds C belong to {0, 1}. We refer to Ptctl^/^ and Ptctl°/^[< 
, >] as the qualitative restrictions of Ptctl and Ptctl [<, >]. 

• Reachability properties are those Pctl properties of the form P[xi^(Fa) or -iP|x](^(Fa). 
Qualitative reachability properties are those reachability properties for which C G {0, 1}. 

The size |$| of a Ptctl formula $ is defined in the standard way as the number of 
symbols in with each occurrence of the same subformula of <I> as a single symbol. 

We now define the satisfaction relation of Ptctl for discrete TMDPs. Given the infinite 
path cv = So '^°''^°> si ■ ■ ■ of the discrete TMDP T, let DiscDur(w,i) = ^o<fc<i 

the accumulated duration along uj until (i + l)-th state. 

Definition 3.2. Given a discrete TMDP J = {S,s, ^ , lab) and a Ptctl formula <I>, we 
define the satisfaction relation |=x of Ptctl as follows: 

s \=j a iff a G lab(s) 

s \=j A $2 iff s \=j <i>i and s \=j ^2 

s iff s 

s \=j iff Probficv G Pathf^i{s) | V^} cxi C, G Adv 

UJ \=j <I>iU^c^2 iff 3i G N s.t. a;(i) |=T (/'2, DiscDur(u;, i) ~ c, 

and \=j 01, yj <i . 

We proceed to define the satisfaction relation of Ptctl for continuous TMDPs. Given 
the infinite path uj = sq '^°''^°> si — U • • • of the continuous TMDP T, let CtsDur(u;, i, d) = 
d + J2o<k<i be the accumulated duration along uj until position (i, d). 

Definition 3.3. Given a continuous TMDP T = {S,s, ^ , lab) and a Ptctl formula 
we define the satisfaction relation of Ptctl as in Definition [3?2l except for the following 
rule for ^i\Jr^c^2- 

^ |=T *l*iU^c*I'2 iff 3 position of uj s.t. uj{i,5) \=j (j)2, CtsDur{uj,i,6) ~ c, 

and uj{j,6') \=j 0i, V positions {j,6') oi u s.t. {j,6') -<^^ (i,5) . 

When clear from the context, we omit the T subscript from |=x. We say that the TMDP 
T = (5, s, — > , lab) satisfies the Ptctl formula denoted by T |= if and only if s |= 
Furthermore, the PTA P satisfies denoted by P |= if and only if T[P] |= $. 
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Complexity of Ptctl model checking for PTA. Given an arbitrary structurally non-Zeno 
PTA P, model checking Ptctl formulae is in EXPTIME |KNSS02| (the algorithm consists 
of executing a standard polynomial-time model-checking algorithm for finite-state proba- 
bilistic systems jBdA951 IBK98 ] on the exponential-size region graph of P). The problem 
of model checking qualitative reachability formulae of the form -iP<i(Fa) is EXPTIME- 
hard for PTA with an arbitrary number of clocks |LS07| . Hence Ptctl model checking for 
structurally non-Zeno PTA with an arbitrary number of clocks is EXPTIME-complete. 

Example 3.4. Consider the PTA P of Figure [TJ The formula P>o(F<9error) holds for the 
configuration {init,0): for every non-deterministic choice, the probability to reach error 
within 9 time units is strictly positive. The formula P<o.i(F<6error) does not hold for 
{init,0): if the adversary chooses to delay until x = 5.4 in wait, and then performs the 
probabilistic edge with the guard 5 < a; < 6, then the probability to reach error is 0.2. 
Note also that the formula P>o.i(F<e error) is not true either in {init,0): the adversary can 
choose to delay in wait until x = 7.8 and then perform the second probabilistic edge, in 
which case the probability to reach error within 6 time units is zero. 

4. Model Checking One-Clock Probabilistic Timed Automata 

In this section we consider the case of IC-PTA. We will see that model checking PCTL 
and Ptctl°/^[<, >] for IC-PTA is P-complete, but remains EXPTIME-complete for the 
logic Ptctl°/^ 

4.1. Model Checking Pctl on IC-PTA. First we present the following result about 
the model checking of Pctl formulae. 

Proposition 4.1. The Pctl model- checking problem for IC-PTA is P-complete. 

Proof. The problem is P-hard because model checking formulae of the form -iP<i(Fa) in 
finite MDPs is P-hard [PT87]. Here we show P-membership. For this we adapt the encoding 
for showing NLOGSPACE-membership of reachability in one-clock timed automata ^LMS04] 
in order to obtain an untimed MDP which is polynomial in the size of the IC-PTA. This 
untimed MDP is then subject to the established polynomial-time Pctl model-checking 
algorithm [BdA95| . 

Let P = {L,l, {x}, inv, prob,C) be a IC-PTA. A state of P is a control location and a 
value V for x. The exact value of x is not important to solve the problem: we just need to 
know in which interval (with respect to the constants occurring in the guards and invariants 
of P) is X. Let Cst(P) be the set of integer values used in the guards and invariants of P, and 
let B = Cst(P)U{0}. We use bo, bi, . . . ,bk to range over B, where = 6o < < • • • < ^fc and 
|B| = k+1. The set B defines a set 2b of 2(A;+1) intervals [bo; bo], {bo; bi), [6i; • • • , (6^, oo). 
We also define a total order on the set 2b, where [6o; bo] < {bo; bi) < [6i; 6i] < • • • < {bk, oo). 
The configuration {l,v) is then encoded by the pair {l,n{v)) such that v belongs to the 
n(f)-th interval in 2b: note that the length of the binary representation of the number of 
an interval is log(2(A; + 1)). We then build an untimed MDP M[P] whose states are the pairs 
{l,n{v)) and the transitions simulate those of P. Note that we can easily decide whether a 
guard is satisfied by the clock values of the n(f )-th interval. A step of P from (/, v) consists 
in choosing a duration d and a distribution fi (as represented by the transition ((/, v),d, fi)), 
and finally making a probabilistic choice. Such a step is simulated in M[P] by a transition 
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{{I, n{v)), u), which corresponds to choosing the appropriate interval n{v + d) in the future 
(i.e., n{v + d) > n(v)), then making a probabihstic choice according to the distribution v 
from {l,n{v + d)), where ,n{v')) = //(/', u') for each state {l',v') of T[P]. 

For a clock constraint £ C!C{{x}), let fipj = {f € M>o | v \= ip}. For an interval 
/ C M>o, let I[{x} := 0] = [0; 0] and /[0 := 0] = I. The MDP for POTL of the PTA P is 
the untimed MDP M[P] = (5m,smi Mi ^«^m) where: 

• Sm = {{l,B) \ I (£ L,B (£Im and B C linv{l)j} and sm = (l [0,0]); 

• — 5- M is the least set such that ((/, B), v) G ^ m if there exists an interval B' € Xb and 
a probabilistic edge {l,g,p) € proh such that: 

(1) B' >B,B'<Z y , and B" C linv{l)l for all B < B" < B'; 

(2) for any {X,l') £ {{x},0} x L, we have that p{X,l') > implies {B' n lg})[X := 0] 
C linvini, 

(3) for any (/', B") G 5m, we have that 1^(1', B") = , B")+ub'{1', B"), where uo{l' , B") = 
pi{x},l') if B" = [0,0] and vq{1\B") = otherwise, and where vb'{1',B") = p{%,l') 
if B' = B" and vb'{1',B") = otherwise. 

• lahy\ is such that lahy\{l,B) = C{1) for each state (/,-B) € Sy\. 

Given a Pctl formula <I> and a state of T[P], we then have that |=t[p] ^ 

if and only if {l,n{v)) ^m[P] which can be shown by induction on the length of the 
formula. The cases of atomic propositions and boolean combinators are straightforward, 
and therefore we concentrate on the case of a formula P[x]a(^iU<I>2)- We can show that, 
for each adversary A of T[P], it is possible to construct an adversary A' of M[P] such 
that, for each state {l^v) of T[P], we have Pro6^^^{u; € Pathf^i{l,v) \ to |=t[p] <&iU<I>2} = 
Prob^i ^^^-^^{lo G Pathf^i{l,n{v)) \ u) \=m[p] ^iU$2}- Conversely, we can show that, for 
each adversary A of M[P], it is possible to construct an adversary A' of T[P] such that, 
for each state {l,v) of T[P], we have Pro^^^^^^^jo; G Path^i{l,n{v)) \ uj |=m[p] $iU$2} = 
Proh^l^^{uj G Pathf^i{l,v) \ uj \=j[p] <I>iU$2}- By the definition of the semantics of Pctl, 
given we have {l,v) ^t[p] Pma(^'iU^>2) if and only if {l,n{v)) ^m[p] PtxA(^'iU^>2). 

The size of M is in 0(|P| • 2 • \&\) and [B] is in 0(2 • \proh\). Because Pctl model 
checking is polynomial in the size of the MDP [BdA95] . we have obtained a polynomial- 
time algorithm for Pctl model checking for PTA. □ 

4.2. Model checking Ptctl'^/^[<, >] on IC-PTA. In this section, inspired by related 
work on discrete-time concurrent game structures [LMU06] . we first show that model- 
checking Ptctl''/^[<, >] properties of discrete TMDPs can be done efficiently. Then, in 
Theorem l4.3l using ideas from the TMDP case, we show that model checking Ptctl"/^ [<, >] 
on IC-PTA can also be done in polynomial time. 

Proposition 4.2. Let T = (S", s, — > , lab) he a structurally non-Zeno discrete TMDP and $ 
be a Ptctl'^/^ [<, >] formula. Deciding whether T \= ^ can be done in time 0{\^\-\S\-\^\). 

Proof sketch. The model-checking algorithm is based on several procedures to deal with 
each modality of Ptctl''/^[<, >]. The boolean operators and the Pctl modalities (without 
timed subscripts) can be handled in the standard manner, with the Pctl properties verified 
on the untimed MDP T" corresponding to T. For formulae P[xi^(<I>iUr^c*l?2)) we assume that 
the truth values of subformulae $i and <I>2 are known for all states of T. First, given that 
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the TMDP is structurahy non-Zeno, we have the equivalences: 

P<0($lU^c^2) = -E($iU^c«'2) 
P>l($lU<e$2) = A($iU<e^'2) 
P>l($lU>e$2) = A($iU>e(P>l($lU^>2))) 

where E (respectively, A) stands for the existential (respectively, universal) quantifica- 
tion over paths which exist in the logic Tctl. Thus we can apply the procedure pro- 
posed for model checking Tctl formulae - running in time Od^j • | — > |) - over weighted 
graphs [ LMS05] (in the case of P>i($iU>c$2), by first obtaining the set of states satisfying 
P>i(<I>iU^>2), which can be done on T" in time 0(|Edges( — > )|), where |Edges( ^ )[ = 
E(M,i')G^|support(z^)|). 

The problem of verifying the remaining temporal properties of Ptctl'^/^[<, >] can 
be considered in terms of turn-based 2-player games. Such a game is played over the 
space S U — > , and play proceeds as follows: from a state s G S, player P„ (representing 
nondeterministic choice) chooses a transition (s, d, z^) G — > ; then, from the transition 
(s,(i, z^), player Pp (representing probabilistic choice) chooses a state s' G support(i/). The 
duration of the move from s to s' via (s, d, v) is d. Notions of strategy of each player, and 
winning with respect to (untimed) path formulae of the form <I>iU<I>2, are defined as usual 
for 2-player games. 

For the four remaining formulae, namely P[>cjf (<I>iU^c'&2) for CXK^ G {> 0, < 1}, and 
~G {<,>}, we consider the functions a, (3,^,6 : S N, for representing minimal and 
maximal durations of interest. Intuitively, for a state s G 5, the value a{s) (respectively, 
7(5)) is the minimal (respectively, maximal) duration that player Pp can ensure, regardless 
of the counter-strategy of Pn, along a path prefix from s satisfying $iU$2 (respectively, 
<I>iU(P>o(^iU<I>2))). Similarly, the value /?(s) (respectively, 6{s)) is the minimal (respec- 
tively, maximal) duration that player P„ can ensure, regardless of the counter-strategy of 
Pp, along a path prefix from s satisfying $iU$2 (respectively, <I>iU(-iP<i(<I>iU<I>2))). 

If there is no strategy for player Pp (respectively, player P„) to guarantee the satisfaction 
of <I>iU<^2 along a path prefix from s, then we let a{s) = 00 (respectively, /3(s) = 00). 
Similarly, if there is no strategy for player Pp (respectively, player Pn) to guarantee the 
satisfaction of $iU(P>o(<I>iU<I>2)) (respectively, $iU(-iP<i($iU<I>2))) along a path prefix 
from s, then we let 7(5) = —00 (respectively, 6{s) = —00). 

Using the fact that the TMDP is structurally non-Zeno, for any state s € S, we can 
obtain the following equivalences: 

• s \= P>o(<I'iU<c<I>2) if and only if a{s) < c; 

• s ^ P<i(<I>iU<c$2) if and only if /3{s) > c; 

• s \= P>o($iU>c<I>2) if and only if 7(5) > c; 

• s \= P<i(<^iU>c<I>2) if and only if 6{s) < c. 

The functions a, f5, 7, 6 can be computed on the 2-player game by applying the same methods 
as in |LMO06j for discrete-time concurrent game structures: for each temporal operator 
IP'Mc(^iUr^c$2), this computation runs in time 0{\S\ -1^1). We decompose the proof into 
the following four cases, which depend on the form of the formula to be verified. 

<I> = P>o(<I'iU<c$2)- To compute the value a{s), we introduce the coefficients a*(s) defined 
recursively as follows. Let a^{s) = if s |= $2) let a^{s) = 00 otherwise, and let: 
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if s ^ ^>2 

oo if s ^ -i^i A -■<I>2 

max {d+ min {a*(s')}} if s ^ $1 A -■$2- 

{s,d,u)G^ s'Gsupport(i/) 



Fact 1. If a*(s) < 00, the value a*(s) is the minimal duration that player Pp can ensure 
from s with respect to <l>iU<l>2 in at most 2i turns . If a*(s) = 00, player Pp cannot ensure 
<I>iU<I>2 in 2i turns. 

Proof of FactUl The proof proceeds by induction over i. The result is immediate for i = 0. 
Now assume the property holds up to i 

Consider a^~^^{s). The cases for a'^~^^{s) = 0, and a*"'"^(s) = 00 with s \= -i<I>i A -'$2, 
are trivial. Now assume a^~^^{s) = 00 and s ^ A -i<I>2: by the definition of a*^"^(s), there 
exists a transition z/) from s such that any possible successor s' S support(z^) verifies 
a*(s') = cxD. By the induction hypothesis this entails that there is no strategy for Pp to 
ensure <I>iU<I>2 in less than 2i turns from any s' E support(z^), and then there is no strategy 
for Pp from s for games with 2(i + 1) turns. 

Assume a*^^(s) G N. Let 9 be the minimal duration that player Pp can ensure with 
respect to ^>iU^>2, for games with at most 2(i + 1) turns. This duration 9 is obtained from a 
choice of transition (s, d, v) of P„ and a choice of state s' G support(z^) of Pp, where, by the 
induction hypothesis, we have = d+a^{s'). We also have that this s' is the best (minimal) 
choice for Pp among all states in support(z^); that is, a*(s') = 'cai'^s" &support{v){'^^ i^")} ■ Given 
the definition of a*^^(s), we have that a^^^{s) equals: 

max {d' + min {a\s")}} >{d+ min {a\s")]} = d + a\s') = 9 , 

{s,d' ,u')G^ s"Gsupport(i'') s"Gsupport(i^) 

However, as 6 corresponds to the best (maximal) choice for P„, we cannot have a*^^(s) > 9, 
and therefore a^'~^^{s) =9. □ 

We claim that al'^l(s) = a{s). First note that we clearly have al'^l(s) > a{s). Now 
assume a{s) < al'^l(s): this value a{s) is obtained by a strategy (for Pp) that uses more 
than 2151 turns. Therefore, along some path generated by this strategy there will be at 
least one occurrence of a state s'. However, as the TMDP is structurally non-Zeno, this 
loop has a duration strictly greater than 0, and it can be removed by applying earlier in the 
path the last choice done for state s' along the patlQ- Such a looping strategy is clearly not 
optimal for Pp and need not be considered when computing a{s). Hence the computation 
of a''^l, and thus a, can be done in time 0{\S\ -1^1). 

<I> = P>o(<I>iU>c<I*2)- In order to establish the set of states satisfying we first compute the 
sets of states satisfying two untimed, auxiliary formulae. The first formula we consider is 
IF'>o(*I*iU<I*2): obtaining the set of states satisfying this formula relies on qualitative Pctl 
analysis of the underlying untimed MDP T" of T, which can be done in time 0(|Edges( — > )|). 
The second formula we consider is P>o(^'iU-^<I>2)5 where, for any infinite path to G Pathf^i, 
we have to \= $iU-^<I>2 if and only if there exists i > 1 such that a;(i) |= $2) and uj{j) \= 
for all j < i. The set of states satisfying P>o($iU-^^>2) can be obtained through a combi- 
nation of the usual "next" temporal operator of PCTL (see |HJ94tlBdA95] ) and the formula 
P>o(<I>iU<I>2), and can be computed in time 0([Edges( )|). 



-'^Note that as q(s) 7^ 00, the path induced by the strategy of player Pp is finite. 
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We then proceed to compute, for each state s of T satisfying P>o($iU<I>2), the maximal 
duration 7(5) that player Pp can ensure with respect to <I>iU(P>o($iU$2))- We compute 7 
using the following recursive rules: 

r -00 if s ^ -P>o($iU$2) 
70(5) = <^ if S ^P>o($lU$2) A-P>o($iU^i$2) 

[ 00 if S ^P>o($lU^l$2) 

' -00 if s ^ ^P>o($iU$2) 

y+i(s) = J if s ^P>o($iU$2) A-P>o($iU^i$2) 

I min {d+ max {i\s')}] if s ^ P>o($iU^i$2) 

We have the following fact, the proof of which is similar to that of Fact [TJ 

Fact 2. If —00 < 7*(s) < 00, then 7*(s) is the maximal duration that player Pp can ensure 
from s with respect to <l>iU(P>o(<&iU<I>2)) in at most 2i turns . If 7*(s) = 00 (respectively, 
7*(s)) = —00), then player Pp can ensure P>o(<I>iU-^<l>2) continuously during 2i turns 
(respectively, cannot ensure $iU$2)- 

Proof of Fact\^ Consider 7*^-^(5). The cases for 7*'^"'^(s) = 0, and 7*+"'^(s) = —00 are 
immediate. 

Assume 7*'^^(s) = 00. Then for any distribution from s, there is a probabilistic choice 
leading to some s' with 7*(s') = 00. By the induction hypothesis, we deduce that player Pp 
can ensure P>o(<5iU-^^>2) during 2(i + 1) turns from s. 

Assume 7*"'"^(s) € N. Let be the maximal duration that player Pp can ensure with 
respect to ^>iU$2) for games with at most 2{i + 1) turns. This duration is obtained from 
a choice of {s,d,v) of P„ and a choice of s' € support(z^) of Pp, where, by the induction 
hypothesis, we have 9 = d + 7*(s'). We also have that this s' is the best (maximal) choice 
for Pp among all states in support(i/); that is, 7*(s') = max5»£5uppo|.t(i/){7*(s")}- We have 
that 7*"'"^(s) equals: 

min {d' + max W{s")}} <{«!+„ max W{s")}} = d + f (.') = 9 . 

(s,a )£— > s"Gsupport(i/') s"£support(i/) 

However, as 9 corresponds to the best (minimal) choice for P„, we cannot have 7*^^(s) < 9, 
and therefore 7*'''^(s) = 9. □ 

As in the case of the function a, we claim that 7l'^l(s) = 7(5). We clearly have 
7''^'(s) > 7(5) (indeed we can prove by induction over i that 7*(s) > 7(5) for any i > 0). 
Assume that ■y{s) < 71*^1(5); then as in the case of a, the value 7(5) is obtained by a strategy 
for Pp which generates a path whose length is greater than 15*1 along which a state is visited 
twice. The assumption of structural non-Zenoness means that, if the strategy can choose 
to repeat s' an arbitrary number of times, the elapsed duration along the path becomes 
arbitrarily large and 7(5) = 7l'^l(s) = 00. Hence, there is no need to explore further the 
path. Therefore the computation of 7l'^l, and thus 7, can be done in time OdS*! • | — > |)- 

$ = P<i(<I>iU<c*l*2)- This case can be treated in a similar manner as the case of <1> = 
IP>o(^iU<c*I*2)- Here we aim at computing the minimum duration /3(s) that player P„ can 
ensure with respect to <^iU<^2- Then <I> holds for s if and only if /3(s) > c. We compute the 
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following values with (3^{s) = if s \= ^2, /3^{s) = oo otherwise, and: 

if s h ^>2 

00 if s ^ -^^i A -i^s 

mill {d + max otherwise. 



Fact 3. If /3*(s) < 00, the value /?*(s) is the minimal duration that player P„ can ensure 
from s with respect to $iU$2 in at most 2i turns . If (3^{s) = 00, player P„ cannot ensure 
<I>iU<I>2 in 2i turns. 

The proof of Fact [3] proceeds in a similar manner to that of Fact [H but with the roles of 
players P„ and Pp reversed, and therefore we omit it. Furthermore, we have /3l^l(s) = p{s) 
for similar reasons that we had a''^' = a{s) (again, with the roles of Pn and Pp reversed), 
and hence the computation of (3 can be done in time Od^l • | — > |). 

$ = P<i(<I>iU>c<I>2)- This property is true when player P„ has no strategy to ensure 
<I>iU>c<&2- Similarly to the case of P>o(^iU>c<I'2)i we first compute the sets of states 
satisfying two untimed formulae, namely P<i(<I>iU<I>2) and P<i(<I>iU-^$2)! the complexity 
of which is in 0(|Edges( ) | Y^|Edges(^r)| ) |CJH03j . We then compute, for each state s 
of T satisfying -iP<i(<I>iU<I>2)) the maximal duration 6{s) that player P„ can ensure with 
respect to <I>iU(P<i(-i<I>iU<I>2)). Then s |= $ if and only if 5{s) < c. We compute 5 using 
the following recursive rules: 

(00 if s ^ ^P<i($iU^i«>2) 
5°(s) = < if s ^ ^P<i($iU$2) AP<i($iU^i«>2) 
[ -00 if s ^P<i($iU$2) 

-00 if s ^P<i($iU$2) 

if s ^ ^P<i($iU$2) AP<i($iU^i$2) 

max {d+ min {S\s')}} if s ^ ^P<i($iU^i$2) 



xi+1 



is) 



Fact 4. If —00 < (5*(s) < 00, then (5*(s) is the maximal duration that player P„ can ensure 
from s with respect to $iU(P>o(^iU$2)) in at most 2i turns . If 5*(s) = 00 (respectively, 
(5*(s) = —00), then player P„ can ensure -iP<i(<I>iU-^<I>2) during 2i turns (respectively, 
cannot ensure $iU(-iP<i(<I>iU<I>2))) from s. 

We can adapt the reasoning used in Fact [2] to prove this fact (as in the case of Fact [3]) . 
Finally, with similar reasoning to that used in the case of P>o($iU>c<I>2), we can show that 
(5''^l(s) = 6{s), and therefore S can be computed in time 0(151 • | — |). 

Finally we obtain an algorithm running in time 0(|<I>|-|S'|-|^|). □ 

We use Proposition 14.21 to obtain an efficient model-checking algorithm for IC-PTA. 

Theorem 4.3. Let P = {Lj,X,inv,prob,C) be a IC-PTA and ^ he a Ptctl°/^[<, >] 
formula. Deciding whether P \= ^ can he done in polynomial time. 

Proof sketch. Our aim is to label every state (?, v) of T[P] with the set of subformulae of ^ 
which it satisfies (as \X\ = 1, recall that is a single real value). For each location I G L and 
subformula ^' of we construct a set Sat[Z, ^] C M>o of intervals such that v G Sat[/, if 
and only if {l,v) ^ We write Sat[Z,^'] = Uj=i,...,fc(ci; c^) with (e {[, (} and ) G {], )}. We 
consider intervals which conform to the following rules: for 1 < i < A;, we have Cj < c'j and 
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Cj, c'j € NU{oo}, and for 1 < j < A;, we have c'j < Cj+i- We will see that |Sat[/, ^][ - i.e., the 
number of intervals corresponding to a particular location - is bounded by |^'| • 2 • |pro6|. 

The cases of obtaining the sets Sat[^, for boolean operators and atomic propositions 
are straightforward, and therefore we concentrate on the verification of subformulae ^ of 
the form P[xi(^(<I>iU^c*^*2)- Assume that we have already computed the sets Sat[_, _] for <I>i 
and $2. Our aim is to compute Sat[/, for each location / E L. 

There are several cases depending on the constraint "ixi C". The equivalence 
P<o($iU^c^2) = (E$iU^c^2)) which holds from the structural non-Zenoness property, 
can be used to reduce the "< 0" case to the appropriate polynomial-time labeling procedure 
for -1 (E<I>iU^c*J'2) on one-clock timed automata [LMS04] . where the IC-TA is obtained by 
converting the probabilistic choice of prob to nondeterministic choice. In the "> 1" case, 
the equivalence P>i(<I>iU^c^2) = A ($iU^c(IP>i(^iU$2))) relies on first computing the 
state set satisfying P>i($iU$2)) which can be handled using a qualitative Pctl model- 
checking algorithm, applied to a discrete TMDP built from P, Sat[/,<I>i] and Sat[/,$2]) in 
time 0(|P| • \prob\- + |$2|)), and second verifying the formula A (<I>iU^c(IP>i('&iU<I>2))) 
using the aforementioned method for one-clock timed automata. 

For the remaining cases, our aim is to construct a (finite) discrete TMDP T^' = {S^ ,-, 
— lab^), which represents partially the semantic TMDP T[P], for which the values of the 
functions a, (5, 7 and 5 of the proof of Proposition 14. 21 can be computed, and then use these 
functions to obtain the required sets Sat[_, (the initial state of T*" is irrelevant for the 
model-checking procedure, and is therefore omitted). The TMDP T^' will take a similar 
form to the region graph MDP of PTA [KNSS02j . but, as in the case of the MDP M[P] 
constructed in the proof of Proposition 14. H will be of reduced size. More precisely, the size 
of T^ will be independent of the magnitude of the constants used in invariants and guards, 
and will ensure a procedure running in time polynomial in |P|. 

We now describe the construction of T*". In the following we assume that the sets 
Sat[Z,<I>j] contain only closed intervals (and possibly intervals of the form [6; 00)) and that 
the guards and invariant of the PTA contain non-strict comparisons: the general case is 
explained in Appendix lAl 

Formally we let C = {0} U Cst(P) U UiG{i,2} UzeL Cst(Sat[/, <I>i]), where, as in the proof 
of Proposition 14. H Cst(P) is the set of constants occurring in the clock constraints of P, 
and where Cst(Sat[/, <I>j]) is the set of constants occurring as endpoints of the intervals in 
Sat[/, <I>j]. Moreover for any right-open interval [6; 00) occurring in some Sat[/, _] we add the 
constant & + c -|- 1 to C. We enumerate C as bo,bi, ...,bM with bo = and bi < for 
i < |C|. Note that |C| is bounded hy 4-\^\- \prob\. 

State space of T^: We consider first the definition of S^, the state space of T''. Considering 
the discrete TMDP corresponding to T[P] restricted to states {l,bi), with bi G C, is 
sufficient to compute the values of functions a, f3, 7 and 5 in any state (/, bi). However, this 
does not allow us to deduce the value for any intermediate states in {bi; 6^+1): indeed some 
probabilistic edges enabled from bi may be disabled throughout the interval {bi;bi+i). 
Therefore, in T*", we have to consider also {l,bf) and {l^b^^i) corresponding respectively 
to the leftmost and rightmost points in (6j;6j_|_i) (when i < M). Then S"^ is defined as 
the set including the pairs {l,bi) with 6j G C and bi \= inv{l), and {l,bf) and {l,b^_^i) 
with bi G C, i < M and (bi; 6j+i) C [oti)(Z)|. Note that the truth value of any invariant is 
constant over such intervals (bi; ftj+i). Moreover note that all T[P] states of the form (/, v) 
with V £ {bi;bi-^i) satisfy the same boolean combinations of and $2) and enable the 
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same probabilistic edges. For any {l,g,p) € prob, we write bf \= g (and 6^^]^ \= g) when 
C [[(jf|. Similarly, we write b'l \= inv{l) (and b~j^^ \= inv(l)) when C 
[[in?;(/)]]. For an interval / C ]R>o, we write b^ ^ I and b^j^^ € / when (6j; hj+i) C /. We 
also consider the ordering bo < b^ < b^ < hi < b^ < ■ ■ ■ < < bM < 
Transitions of T^: We now define the set — of transitions of y as the smallest set such 
that {{I, X),d,iy) where A G {b~ ,bi,b^} for some hi E C, if there exists A' > A, 

where A' G {bj, bj, b^} for some bj G C, and {l,g,p) G prob such that: 
9 d = bj - bi, X' \= g, and both A" |= inv{l) and A" C Sat[/,<i>i] \ Sat[^,<i>2] for any 
A < A" < A'; 

• for each {X,l') G support(p), we have \= inv{l') \i X = {x}, and A' |= inv{l') li X = %; 

• for each (/',A") G 5^ we have u{l',\") = vq{1',\") + ux{l',X"), where z^o(^',A") = 
pil',{x}) if A" = [0,0] and iyo{l',X") = otherwise, and i^xil'^ ) = p{l\^) if A" = A' 
and vx[l\X") = otherwise. 

Labelling function of y: To define laV\ for a state {l,bi), we let a^. G lab^'{l,bi) if and 
only if 6j G Sat[Z, ^j], for j G {1, 2}. The states (/, bf) and (/, are labeled depending 
on the truth value of the <I>j's in the interval (bi; fej+i): if (6,; bi^i) C Sat[/, ^j], then a$^. G 
lab^{l,bf) and 0$^. G lab^ {I , b^^^^) . Note that, given the "closed intervals" assumption 
made on Sat[Z,^'j], we have lab'^{l,bf) C lab^{l,bi) and lab^ {l,b^j^i) C lab^{l,bi). 

Note that the fact that P is structurally non-Zeno means that T'" is structurally non-Zeno. 
The size of T^ is in 0(|P|2 • l^-l). 

Now we can apply the algorithms defined in the proof of Proposition 14.21 and obtain 
the value of the coefficients a, /3, 7 or (5 for the states of T^. Our next task is to define 
functions a, (3,^, 6 : S ^ IK>o, where 5 is the set of states of T[P], which are analogues of 
a, /5, 7 or 5 defined on T[P]. Our intuition is that we are now considering an infinite-state 
2-player game with players P„ and Pp, as in the proof of Proposition 14.21 over the state 
space of T[P]. Consider location / G L. For 6 G C, we have a{l, b) = a{l, b), (5{l, b) = f3{l, b), 
7(/, b) = 7(/, b) and 6(1, b) = S(l, b). For intervals of the form (6^; the functions a and 

6 decrease (with slope -1) throughout the interval, because, for all states of the interval, the 
optimal choice of player P„ is to delay as much as possible inside any interval. Hence, the 
value «(/,!') for u G (6j;6j+i) is defined entirely by a(/, 6^^^) asa{l,v) = a{l,b~^-^) + bi+i—v. 
Similarly, 6{l, v) = 5(1, 6^^^) + 5j+i - v. 

Next we consider the values of (3 and 7 over intervals (6j;6j+i). In this case, the 
functions will be constant over a portion of the interval (possibly an empty portion, or 
possibly the entire interval), then decreasing with slope -1. The constant part corresponds 
to those states in which the optimal choice of player P„, is to take a probabilistic edge, 
whereas the decreasing part corresponds to those states in which it is optimal for player P„ 
to delay until the end of the interval. The value (3{l,v) for v G (6j;6j+i) is defined both by 
^(/,6+) and /3(/,6-+i) as P{l,v) = p{l,bt) if b, < v < h+i - mM) " Pi'^.K+i))^ and as 
P{l,v) = P{l,bl^i) — {v — P{l,bf)) otherwise. An analogous definition holds also for 7. 

From the functions a, (3, 7 and S defined above, it becomes possible to define Sat[/, ^] 
by keeping in this set of intervals only the parts satisfying the thresholds < c, > c, > c 
and < c, respectively, as in the proof of Proposition 14. 2[ We can show that the number of 
intervals in Sat[l, ^] is bounded by 2 • 1^*1 • |pro6|. For the case in which a function a, f3, 7 
or 6 is decreasing throughout an interval, then an interval in Sat[/, <I>i] which corresponds 
to several consecutive intervals in can provide at most one (sub)interval in Sat[/,^], 
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because the threshold can cross at most once the function in at most one interval. For the 
case in which a function /3 or 7 combines a constant part and a part with slope -1 within 
an interval, the threshold can cross the function in several intervals contained in 

a common interval of Sat[/,<I>i]. However, such a cut is due to a guard x > /c of a given 
transition, and thus the number of cuts in bounded by |pro6|. Moreover a guard x < k may 
also add an interval. Thus the number of new intervals in Sat[g, ^] is bounded by 2 • |pro6|. 

In addition to these cuts, any interval in Sat[/, ^2] may provide an interval in Sat[Z, 
This gives the 2 • l^'l ■ |pro6| bound for the size of Sat[^, □ 

Corollary 4.4. The Ptctl''/"'^[<, >] model- checking problem for IC-PTA is P-complete. 

4.3. Model checking Ptctl*^/^ on IC-PTA. We now consider the problem of model- 
checking Ptctl''''^ properties on IC-PTA. An EXPTIME algorithm for this problem exists 
by the definition of an MDP analogous to the region graph used in non-probabilistic timed 
automata verification [KNSS02] . We now show that the problem is also EXPTIME-hard by 
the following three steps. First we introduce countdown games, which are a simple class of 
turn-based 2-player games with discrete timing, and show that the problem of deciding the 
winner in a countdown game is EXPTIME-complete. Secondly, we reduce the countdown 
game problem to the Ptctl''/^ model-checking problem on TMDPs. Finally, we adapt 
the reduction to TMDPs to reduce also the countdown game problem to the Ptctl''/^ 
model-checking problem on IC-PTA. 

A countdown game C consists of a weighted graph (S,T), where S is the set of states 
and T C S X N \ {0} x S is the transition relation. If t = (s,d, s') G T then we say that 
the duration of the transition t is d. A configuration of a countdown game is a pair (s, c), 
where s G S is a state and c G N. A move of a countdown game from a configuration (s,c) 
is performed in the following way: first player 1 chooses a number d, such that < d < c 
and (s,d, s') G T, for some state s' G S; then player 2 chooses a transition (s,d, s') G T of 
duration d. The resulting new configuration is (s', c — d). There are two types of terminal 
configurations, i.e., configurations (s,c) in which no moves are available. If c = then 
the configuration (s,c) is terminal and is a winning configuration for player 1. If for all 
transitions (s, d, s') G T from the state s, we have that d > c, then the configuration (s, c) is 
terminal and it is a winning configuration for player 2. The algorithmic problem of deciding 
the winner in countdown games is, given a weighted graph (S, T) and a configuration (s, c), 
where all the durations of transitions in (S, T) and the number c are given in binary, to 
determine whether player 1 has a strategy to reach a winning configuration, regardless of 
the strategy of player 2, from the configuration (s,c). If the state from which the game 
is started is clear from the context then we sometimes specify the initial configuration by 
giving the number c alone. 

Theorem 4.5. Deciding the winner in countdown games is EXPTIME-comp/ete. 

Proof sketch. Observe that every configuration of a countdown game played from a given 
initial configuration can be written down in polynomial space and every move can be com- 
puted in polynomial time; hence the winner in the game can be determined by a straight- 
forward alternating PSPACE algorithm. Therefore the problem is in EXPTIME because 
APSPACE = EXPTIME. 

We now prove EXPTIME-hardness by a reduction from the problem of the accep- 
tance of a word by a linearly-bounded alternating Turing machine [CKS81] . Let M = 
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■,Q0i1acciQ3iQ\ii ^) be an alternating Turing machine, where S is a finite alphabet, 
Q = Qb U Qv is a finite set of states partitioned into existential states Qg and universal states 
Qvi ^Zo £ Q is an initial state, qacc G Q is an accepting state, and A C QxSxQxSx {L, R\ 
is a transition relation. Let us explain the interpretation of elements of the transition rela- 
tion. Let t = {q, a, q', a', £)) G A be a transition. If machine M is in state q G Q and its 
head reads letter o" G S, then it rewrites the contents of the current cell with the letter a', 
it moves the head in direction D (either left if D = L, or right if D = R), and it changes 
its state to q'. 

Let G > 2 • IQ X S| be an integer constant and let tt; G be an input word. Without 

loss of generality, we can assume that the alternating Turing machine M uses exactly n 
tape cells when started on the word w, and hence a configuration of machine M is a word 
bobi • • • b„_i G (S U Q X S)". Let (•) : (S U Q x S) -> { 0, 1, . . . , G - 1 } be an injection. 
For every aGSuQxS, itis convenient to think of (a) as a G-ary digit, and we can 
encode a configuration u = bobi • • • b„_i G (E U Q x E)" of machine M as the number 

We first define countdown games which have the role of checking the contents of the 
tape; these countdown games will be used as gadgets later in the overall reduction. Let 
z G N, < i < n, be a tape cell position, and let a G EUQ x E. We define a countdown game 
Check*'^, such that for every configuration u = ho ■ ■ ■ b„_i of machine M, player 1 has a 
winning strategy from the configuration (sq^, N{u)) of the countdown game Check*'^ if and 
only if hi = a. The game Check*'*^ has states { Sg^, . . . , Sn^ }, and for every A;, < A; < n, 
we have a transition (s^'^,d, s^'^^) G T, if: 

^_f(a)-G'= if k = i, 
~\{h)-G'' if A;7^zandbGEU5xS. 

There are no transitions from the state sll^. Observe that if b^ = a then the winning strategy 
for player 1 in game Check*'** from N(u) is to choose the transitions (s^!^, b^ • G^, s^'^-j^), for 
all fc, < A: < n. If, however, bj / a then there is no way for player 1 to count down from 
N{u) to in the game Check*'^. 

Now we define a countdown game Cm, such that machine M accepts a word w = 
cToO"! . . . (Tn-i if and only if player 1 has a winning strategy in Cm from configuration 
{qo,N{u)), where u = (qo, ao)ai . . . an-i is the initial configuration of tape contents of 
machine M with input w. The main part of the countdown game Cm is a gadget that 
allows the countdown game to simulate one step of the Turing machine M. Note that one 
step of a Turing machine makes only local changes to the configuration of the machine: if 
the configuration is of the form u = slq . . . a„_i = (Tq . . . (Tj)crj+i . . . Un-i, then per- 

forming one step of M can only change entries in positions i — 1, i, or i + 1 of the tape. For 
every tape position i, < i < n, for every triple r = (o".i_i, (q, dj), G E x (Q x E) x E, 

and for every transition t = (q,a,q' ,a' , D) G A of machine M, we now define the number 
dj'^, such that if ai = a and performing transition t at position i of configuration u yields 
configuration n' = bo . . . b„_i, then N(u) — dl'^ = N{u'). For example, assume that i > 
and that D = L; from the above comment about locality of Turing machine transitions we 
have that b^ = a^ = cTfc, for all fc ^ { i — 1, i, i + 1 } and bj+i = a^+i = (Jj+i. Moreover we 
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have that bi_i = and bj = a' . We define d^'^ as foUows: 

4^ = ((b.-i) - (a.-i)) • G*-i + ((b,) - (a,)) ■ 

The gadget for simulating one transition of Turing machine M from a state q € (5\{gacc} 
has three layers. In the first layer, from a state € Q \ { Qacc }) player 1 chooses a pair (i, r), 
where < i < n, is the position of the tape head, and r = (a, b, c) E S x (Q x S) x S is 
his guess for the contents of tape cells i — and i + 1. In this way the state {q, i, r) of the 
gadget is reached, where the duration of this transition is 0. Intuitively, in the first layer 
player 1 has to declare that he knows the position i of the head in the current configuration 
as well as the contents r = (a, b, c) of the three tape cells in positions i — 1, i, and i + 1. 
In the second layer, in a state {q,i,T) player 2 chooses between four successor states: the 
state {q,i,T,*) and the three subgames Check'-i'**, Check^'^, and Check^+i''^. The four 
transitions are of duration 0. Intuitively, in the second layer player 2 verifies that player 1 
declared correctly the contents of the three tape cells in positions i — and i + Finally, 
in the third layer, if q G Q3 (respectively, q G Qy), then from a state {q,i,T,*) player 1 
(respectively, player 2) chooses a transition t = {q,a,q' ,a' , D) of machine M, such that 
b = {q,a), reaching the state q' £ Q oi the gadget, with a transition of duration dj'^. 

Note that the gadget described above violates some conventions that we have adopted 
for countdown games. Observe that durations of some transitions in the gadget are and 
the duration dj'^ may even be negative, while in the definition of countdown games we 
required that durations of all transitions are positive. In order to correct this we add the 
number to the durations of all transitions described above. This change requires a minor 
modification to the subgames Check*'*^: we add an extra transition (s^*^, G", sll^). We need 
this extra transition because instead of starting from (go, ]^iu)) as the initial configuration 
of the countdown game Cm, where u is the initial configuration of M running on w, we 
start from the configuration (go, G^"' + A^(n)). In this way the countdown game can perform 
a simulation of at least G" steps of machine M; note that G" is an upper bound on the 
number of all configurations of machine M. 

Without loss of generality, we can assume that whenever the alternating Turing ma- 
chine M accepts an input word w then it finishes its computation with blanks in all tape 
cells, its head in position 0, and in the unique accepting state qacc', we write Uacc for this 
unique accepting configuration of machine M. Moreover, assume that there are no transi- 
tions from the accepting state qacc in machine M. In order to complete the definition of 
the countdown game Gm, we add a transition of duration N{uacc) from the state qacc of 
game Cm- □ 

Proposition 4.6. The Ptctl*^/^ model- checking problem for structurally non-Zeno discrete 
TMDPs is EXPTlME-complete. 

Proof. An EXPTIME algorithm can be obtained by employing the algorithms of |LS05j . 
We now prove EXPTIME-hardness of Ptctl'^/^ model checking on discrete TMDPs by a 
reduction from countdown games. Let C = (S,T) be a countdown game and (s, c) be its 
initial configuration. We construct a TMDP Tc.(s^c) = {^,3, — > ,lab) such that player 1 
wins C from (s, c) if and only if Tc^(s^c) N ~'IF'<i(F=ctrue). Let S" = S and s = s. We 
define to be the smallest set satisfying the following: for each s € S and d € N>o, if 
{s,d,s') G T for some s' G T, we have (s,d, z^) G — > , where u is an arbitrary distribution 
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over S such that support(i/) = {s' | (s,d,s') G T}. The labelhng condition lab is arbitrary. 
Then we can show that player 1 wins C from the configuration (s, c) if and only if there 
exists an adversary of Tc^(s,c) such that a state is reached from s = s after exactly c time 
units with probability 1. The latter is equivalent to s ^ -iP<i(F=c'true). □ 

We now show that the proof of Proposition 14. 61 can be adapted to show the EXPTIME- 
completeness of the analogous model-checking problem on IC-PTA. 

Theorem 4.7. The Ptctl°/^ model- checking problem for IC-PTA is EXFTIME- complete. 

Proof. Recall that there exists an EXPTIME algorithm for model-checking Ptctl''''^ prop- 
erties on structurally non-Zeno PTA |KNSS02] : hence, it suffices to show EXPTIME- 
hardness for Ptctl*^/^ and IC-PTA. Let C be a countdown game with an initial config- 
uration (s, c). We construct the IC-PTA P^'^^^^ = {L,l,{x},inv,prob,£) which simulates 
the behaviour of the TMDP Tc^(s.c) of the proof of Proposition 14.61 in the following way. 
Each state s S S of Tc,(s,c) corresponds to two distinct locations ll and l"^ of P^fj^c)- 
Let U = {li \ s £ S} for i G {1, 2}, let L = U L^, and let I = /i. For every transition 
(s,d, z^) G of Tc^(s.c), we have the probabilistic edges {ll,x = 0,p^), {l"^, x = d,p'^) G prob, 
where /g) = 1, and Z^,) = i^{s') for each location s'. For each state s G S, let 

inv(ll) = {x < 0) and inv{ll) = true. Therefore the PTA P^*^^^) moves from the location 
/g to ^3 instantaneously. Locations in are labelled by the atomic proposition a, whereas 
locations in are labelled by 0. Then we can observe that P^lsc) N ~'IP<i(F=ca) if and 
only if Tc_(s^c) N ~']P<i(F=ctrue). As the latter problem has been shown to be EXPTIME- 
hard in the proof of Proposition l4.6l we conclude that model checking Ptctl'^/^ on IC-PTA 
is also EXPTIME-hard. □ 

In Figure O we illustrate the transformation from countdown games to TMDP, then to 
IC-PTA, for a fragment of a countdown game. For simplicity, we omit guards of the form 
X = and invariant conditions of the form true. 

5. Model Checking Two-Clocks Probabilistic Timed Automata 

We now show EXPTIME-completeness of the simplest problems that we consider on 
2C-PTA. 

Theorem 5.1. Qualitative probabilistic reachability problems for 2C-PTA are EXPTIME- 
complete. 

Proof. EXPTIME algorithms exist for probabilistic reachability problems on structurally 
non-Zeno PTA [KNSS02] . and therefore it suffices to show EXPTIME-hardness. We pro- 
ceed by reduction from deciding the winner in countdown games. Let C be a count- 
down game with initial configuration (s, c), and let Pc'^g,,) = {L,l,{x},inv,prob,C) be 

the IC-PTA constructed in the proof of Theorem [121 We define the 2C-PTA P^'^^-^ ^-^ = 
{L U {I*}, I {x, y}, inv', prob', £') fr om P^*^^ in the following way. The set of probabilistic 
edges prob' is obtained by adding to prob the following: for each location Z G L^, we extend 
the set of outgoing probabilistic edges of I with {l,y = c,p^ ), where p^ (0,^*) = 1; we also 
add (/*,true,p' ) to prob' . For each / G L, let inv'{l) = inv{l), and let inv'{l*) = true. 
Finally, we let C'{1*) = a, and C{1) = for all I G L. Then P^^s.c) N -F<i(Fa) if and only 
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Figure 2: Reduction from countdown games 



if P}f(sc) H <i{^ =c(i) ■ The EXPTIME-hardness of the latter problem has been shown in 
the proof of Theorem 14.71 and hence checking qualitative probabilistic reachability proper- 
ties such as ^P<i(Fa) on 2C-PTA is EXPTIME-hard. □ 
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In Figure [2] we illustrate the reduction from countdown games to 2C-PTA (via the 
reduction to TMDPs and IC-PTA). 

Corollary 5.2. The Pctl, Ptctl°/-'^[<, >], Ptctl°/^, Ptctl[<,>] and Ftctl model- 
checking problems for 2C-PTA are 'EXPTYME- complete. 



6. Forward Reachability for One-Clock Probabilistic Timed Automata 



Model-checking tools for non-probabilistic timed automata such as Uppaal BDL"'"06 
are generally based on algorithms for forward reachability through the state space: such 
algorithms start from the initial state and explore the state space by executing transitions 
either in a depth-first or breadth-first manner, and representing sets of clock valuations sym- 
bolically using zones. Forward reachability algorithms can be used for verifying reachability 
properties, such as "the location error is reachable from the initial state". 

We recall that the zone-based forward reachability approach has been adapted for PTA 
by Kwiatkowska et al. |KNSS02] . and can be used to reason about the maximal probability 
of reaching a certain set of locations. More precisely, an (untimed) MDP is constructed by 
exploring the state space of the PTA from its initial state. Then the maximal probability 
of reaching a set of locations is computed on the MDP. The appeal of this approach is its 
practical applicability |DKN04) . A disadvantage of the approach is that, in general, it can 
be used only to obtain an upper bound on the maximal probability of reaching a set of 
locations of a PTA, rather than the actual maximal probability of reaching the locations. 
In particular, Kwiatkowska et al. |KNSS02] present an example of a 2C-PTA in which the 
forward reachability approach does not compute the actual maximal probability of reaching 
a set of locations. 

In this section, we consider the application of the forward reachability approach of 
Kwiatkowska et al. |KNSS 02] to IC-PTA, and show that the maximal and minimal prob- 
abilities computed on the untimed MDP corresponds to the actual maximal and minimal 
probabilities of reaching a set of locations of the IC-PTAS 

First we introduce some notation. Consider the IC-PTA P = (L,l, {x}, inv , prob, C), 
which we assume to be fixed throughout this section. As in the proof of Proposition 14.11 
we use B = Cst(P) U {0} to refer to the set of constants used in the guards and invariants 
of P (and 0). Let Tfr be the set of intervals of the form (6; b'), where 6 € B, 6' G B U {oo}, 
(€ {(, [} and ) E {),]}• The aim of forward exploration is to compute state sets represented 
by pairs of the form (1,1), where Z € L is a location and I € IpR is an interval of the above 
form. The pair {1,1) represents all states {l,v) of T[P] such that v ^ I. 

We define the operator post, which maps a location-interval pair, a probabilistic edge, a 
reset set and a location, to a location-interval pair. Intuitively, post returns the set of states 
obtained after executing a probabilistic edge (including making the probabilistic choice 
concerning the target location and clock reset) and then letting time pass. First consider 
a clock constraint -0 ^ CC[{x}), and recall that [V']] = {f G M>o | v \= -0}- By definition 
M G T^R. For ah /, /' G TpR, note that In/' G TpR. Furthermore, let // = (6; oo)n[[in?;(/)|, 
and recall that I[{x} := 0] = [0; 0] and /[0 := 0] = /. Let (/,/) G LxTpR, let {l,g,p) G prob, 
and let {X,l') G support(p). Then post{{l,I),{l,g,p),X,l') = (/', ((M n/)[X := 0])/,)- 



^Readers familiar with Kwiatkowska et al. |KNSS02] will note that the presentation below is simplified 
with regard to that for PTA with an arbitrary number of clocks. In particular, to ease notation, we consider 
that forward reachability can consider states reached after reaching the target set of locations. 



MODEL CHECKING PROBABILISTIC TIMED AUTOMATA 



23 



We now proceed to define formally an untimed MDP, the states of which are intervals 
of the form /) E L x XpR and which are obtained by forward exploration from the initial 
state of P. The probabilistic transition relation of the untimed MDP is derived from the 
probabilistic edge relation of P. 

Definition 6.1. The forward reachability MDP of the PTA P is the untimed MDP FR[P] = 
('S'frjSfr, FR,^a&FR) where: 

• 'S'fr Q L X 2"fr is the least set of location-interval pairs such that: 

{(MO;0]I)}U U U U po5t{il,I),il,g,p),X,l')CSfR. 

{l,I)£SfR {l,g,p)eprob (X,«')esupport(p) 

• spR = (J^ [0,0] J) is the initial state. 

• ^ FR is the least set such that {{l,I),p) € — > fr if there exists a probabilistic edge 
(I'^g^p) £ P''"ob such that: 

(1) 

(2) for any {X,l') € {{a;},0} x L, we have that p{X,l') > implies {InM)[X := 0] n 
ltnv{l')l + 0; 

(3) for any (/',/') e 5fr, we have that = p^il' ,V) + Piil'J'), where po{l',I') = 
p{{x}, I') if (/', I') = post((^, /), {l,g,p), {x}, I') and po{i', I') = otherwise, and where 
pj{l',I') =p(0,/') if (/',/') = post((/, /),(/, 5,p),0,r) and /)/(/',/') = otherwise. 

• labff^ is such that lab^^{l,I) = C{1) for each state (Z,/) € S'fr. 

We now show that reachability properties can be verified on FR[P]. The overall proof 
of this results proceeds by relating FR[P] to the untimed MDP M[P] of Proposition 14. 1^ 
which we have established can be used to verify reachability properties (because the set 
of reachability properties is a subset of Pctl). Recall the definition of the set of intervals 
2b and the untimed MDP M[P] = (5'm,sM) — ^ Mi lo,bu) of Proposition 14. li We define the 
function Istlnt : Xfr — > Ib in the following way: given / € Tfr, let lstlnt(/) = min{i? € 2b | 
B C /}. We define a restricted version of M[P], namely lst[P] = {SistiSu, — > isti^a^ivi)) 
where 5ist = {{l, lstlnt(/)) | (/,/) £ S'fr}, and where ^ ist ^ — > m is defined as the least 
set such that {{l,B),v) € ist if conditions (1), (2) and (3) of the definition of — > m are 
satisfied, and additionally (4) B = lstlnt(/) for some / S 2fr such that (/,/) G 5fr. The 
untimed MDP lst[P] will be used as an intermediate model to relate FR[P] to M[P]. First 
we consider the relationship between FR[P] and lst[P]. 

Lemma 6.2. (1) For each {{1,1), p) G fR; there exists ((/, lstlnt(/)), z^) G — > ist such 

that, for all {I' J') G 5fr, we have p{l' J') = v{l' , lstlnt(/')). 
(2) For each {1,1) G S'fr, and for each {{I , lst\nt{I)) , u) G ist, there exists {{l,I),p) G 

— > FR such that, for all {I', I') G S'fr, we have v{l' , lstlnt(/')) = p{l',I'). 

Proof. We prove part (1), noting that part (2) can be shown in a similar manner. Let 
{{1,1), p) G FR- Then there exists a probabilistic edge {l,g,p) G prob satisfying the 
conditions of Definition 16. li We identify the transition ((/, Istlnt (/)), i^) G — > ist in the 
following way. Noting that / Pi [[^J 7^ (condition (1) of Definition 16. Ih . we let B = 
lstlnt(/ n \g\). Therefore B > lstlnt(/). Furthermore, we have that B' C Jm?;(/)]] for all 
lstlnt(/) < B' < B, satisfying condition (1) of the definition of M[P] (see Proposition 14. ip . 
Furthermore, condition (2) for — > fr of Definition 16. ll implies condition (2) of the definition 
of M[P]. 
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It remains to show that, for ah {I', I') G 5fr, we have p{l',I') = lstlnt(/')). By 
definition, it suffices to show that for ah (/',/') G 5fr, we have po{l',I') = i^o{l' , Istlnt(I')) 
and pi{l\I') = i/istint(/)(^',lstlnt(I')). 

If {l',r) = po5tl{l,I),{l,g,p),{x},l'), then lstlnt(/') = [0;0], and by definition we 
have poil',1') = Pi{x},l') = i^o(/',lstlnt(/')). If (/',/') / post((/, /),(/, {x}, /'), then 
lstlnt(/') ^ [0;0], and po{l',r) = = i^o(^', lstlnt(/')). 

If {I', I') = post{{l, I), {I, g,p),$, I'), then, by definition of post, we have I' = 
((M n/)[0 := 0])J, = (M □/)]■,. We then conclude that lstlnt(/') = lstlnt(I n {g}). 
Hence, by definition of M[P], we have that ,l5t\nt{I')) = p{$,l'). By Defini- 

tion EH we have pi{l',I') = p(0,Z')> and therefore pi{l',I') = z^istint(/)(^', Istlnt(I')). If 
(/',/') ^ post((/,/),(/,5,p),0,r), then we obtain pi{l',I') = = i^istint(/)(^', lstlnt(/')). 

We conclude that p{l' , I') = v{r , Istlnt(I')) for all /') € 5fr- □ 

We say that two untimed MDPs M" = (^i, si, i, lahi) and M2 = (S2, S2, ^ 2, lah2) 
are isomorphic if there exists a bijection f : Si ^ S2 such that: 

(1) for each state s S 5i, we have lahi{s) = lab2{f{s)); 

(2) fisi) = S2; 

(3) (s, I/) G ^ 1 if and only if (/(s), /(v)) & ^ 2, where /(z/) G Dist(5'2) is the distribution 
defined by /(z^)(s') = zv(/-i(s')) for each s' G S2. 

Lemma 6.3. The untimed MDPs FR[P] and lst[P] are isomorphic. 

Proof. We consider the bijection / : 5fr Sist such that f{l,I) = (Z, lstlnt(/)) for each 
{1,1) G 5fr. First we have that lah^R{l,I) = C{1) = lahu{l, lstlnt(/)). Second we have that 
/(sfr) = /((UO;0]J)) = ([, lstlnt([0;0]J)) = (/~,[0;0]) = su- Third, Lemma [O] establishes 
that ((/,/), p) G ^ fr if and only if ((/, Istlnt(I), /(p)) G ^ ist- □ 

Given that isomorphism is as least as strict as probabilistic bisimilarity [SL95j . and 
that, for any adversary A of an MDP, we can define a corresponding adversary A' of a 
probabilistically bisimilar MDP such that A and A' have the same reachability probabilities, 
we obtain the following corollary. 

Corollary 6.4. Let a G AP . For any adversary A G Adv^^]y^, there exists an adversary 
A' G Advi^iyp^ such that: 

Proh^^iuj G Pathf^iis^^) \ uj ^fr[P] ^a] = Probg^{u: G Pathf^i{sisi) \ ^ Nist[P] Fa}.(6.1) 

Conversely, for any adversary A' G Advi^i^pj, there exists an adversary A G ^d?;FR[p] such 
that Equation \6.1\ holds. 

It remains to relate lst[P] to M[P]. The intuition underlying the following results is the 
following: while lst[P] is a restriction of M[P], the additional transitions of M[P] only result 
in states from which the ability to enable probabilistic edges is weakened. For any two 
states B), {I, B') of M[P], we write (/, B) < {I', B') if I = I' and B < B' . Furthermore, for 
the distribution v G Dist(5ist) and i^' G Dist(S'M), we write ^ z^' if there exists a bijection 
/ : support(z^) support(z/') such that f{v) = u', and, for each {l,B) G support(z^), we have 
B) :< f{l, B). The following lemma can be derived directly from the definitions of lst[P] 
and M[P]. 

Lemma 6.5. Let {l,B) G Sist and {l,B') G Sm be such that {l,B) ^ {l',B'). Then, for 
each {{l,B'),u') G — > there exists {{l,B),u) G — > ist such that v -< v' . 
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Lemma 16.51 then allows us to construct, for any adversary A of M[P], an adversary 
A' of lst[P] such that the probability of reaching a given set of locations from the initial 
state is the same for A and A' (this fact also follows by noting that (^)~^ is a probabilistic 
simulation |SL95j ). The converse result, which states that, for any adversary A of lst[P], 
an adversary A' of M[P] such that the probability of reaching a given set of locations from 
the initial state is the same for A and A', follows from the fact that lst[P] is a restriction 
of M[P]. We then obtain the following corollary. 

Corollary 6.6. Let a G AP. For any adversary A G Advi^^^p^, there exists an adversary 
A' € Advf^/\[p^ such that: 

Probf^JuJ G Pathf^iisist) \ UJ hist[P] fa} = Probf^{u G Pathf^i{sM) \ w ^mIP] Fa} . (6.2) 

Conversely, for any adversary A' G Adv^^p-^, there exists an adversary A G Adv ist[p] such 
that Equation \6.S\ holds. 

Combining Corollary 16.41 and Corollary 16.61 and using the proof of Proposition 14. 1^ 
which states that the results of model checking a Pctl formula (including reachability 
properties of the form P^A(Fa)) on M[P] correspond to the satisfaction of the formula on 
T[P], we conclude with the following corollary. 

Corollary 6.7. Let a G AP, ~G {<, <, >, >} and X G [0, 1]. We have FR[P] \= P^A(Fa) if 
and only if T[P] \= P^A(Fa). 



7. Conclusion 

We have shown that probabilistic model-checking problems for IC-PTA can be per- 
formed efficiently if qualitative properties with non-punctual timing bounds are considered. 
If the temporal logic features punctual timing bounds, the problem becomes EXPTIME- 
complete. We have also shown that the forward reachability algorithm of Kwiatkowska 
et al. |KNSS02] can be used to compute the exact probability of reaching a state set for 
IC-PTA. For future work, we intend to consider the complexity of model checking IC- 
PTA against quantitative properties without punctual timing bounds (that is, properties of 
Ptctl[<,>]). On the other hand, we have shown that model-checking problems for 2C- 
PTA are EXPTIME-complete, regardless of the probability threshold and timing bounds 
used. 
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Appendix A. Model checking Ptctl°/^[<, >] over PTAs with strict 

CONSTRAINTS 

Here we describe briefly the general case for the model-checking algorithm of Theo- 
rem |3]3l that is when the guards and invariants of P may be strict and when the intervals in 
Sat[/, <I>j] may be open (or half-open). This makes the algorithm more difficult to describe 
even if the complexity remains polynomial. Here we will only give the main idea about how 
to deal with these kind of constraints. 

First note that an optimal strategy of either of the players Pn or Pp cannot always be 
restricted to perform transitions at integer points: if a transition has to be performed as 
soon as possible and if it has a guard x > d, then it is not possible to perform it from the 
position d, and in some cases it is not optimal to wait until d-\-l. In fact, sometimes there 
is even no optimal strategy corresponding to the optimal values (for a, (3, 7 and 5). The 
same remark holds for the notion of optimal (timed) path in timed automata |ATP04j . We 
have to define the optimal value as a constant k such that there exist strategies with a cost 
arbitrarily close (above or below) to k. Thus the optimal value will be denoted as "e /c" 
with e € {<,=,>}. For example, "< 2" will mean that the optimal value is less than 2 but 
arbitrarily close to 2. 

The method proposed for the simple case has to be modified in order to handle the 
(non)strict value. For each Ptctl''/"^[<, >] modality, we can use a variant of the finite 
discrete TMDP T*^ defined in the proof of Theorem 14.31 again we consider the singular 
states {I, hi) and the "symbolic states" (Z, (5^; ftj+i)) with bi € B, with the two special 
positions bf and h^j^^. 

Consider the case of subformulae of the form P>o(<I'iU<c*l*2)- Then we want to compute 
the function a for any configuration {l,v) of T[P]. Figure [3] shows two simple examples 
where the value for a is indicated for every integer point and for the left and right side 
of the intervals. Note that in these examples, we just assume that proh contains the two 
probabilistic edges {l,x > l,p) (respectively, {l,x = 2,p)) where p({x}, /'), and {l',x = l,p') 
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where p'{{x},l"). Moreover the only state satisfying $2 is (^",0), and all states satisfy $1. 
The value a corresponds to the duration between the current state and (/", 0). This example 
is sufficient to illustrate the problem of strict and non-strict values. 

Let us consider the structure of the function a. For the singular points (Z, bi) the value 
can be of the form "< k" , "= /c", "> k" , or 00 when there exists a strategy for P„ to 
avoid $2 forever. Note that the case "> k" can occur for a state {l,bj) when the property 
$2 holds for an interval (/, (ftj; ftj+i)): reaching this interval from {l,bj) can be done by a 
duration strictly greater than bi — bj . The other cases are illustrated on Figure [3l 

<3 <2 =3 =2 =1 



<3 >2 I <2 >i 







l<3 >2 


<2 >1 1 




Figure 3: Example of optimal value for a 

Now consider the case of symbolic states (/, {bi] The structure of a over such an 

interval is always decreasing: indeed either the best strategy for P„ consists in performing 
a distribution from the current interval, in which case it is always better to delay until the 
last point (b^^i) of the interval, or the best strategy consists in delaying until a future state 
or interval. We can see that the value of the rightmost position inside the interval will be 
always of the form "> k": indeed it depends either on the value in (if the strategy 
goes through this point) or on the value in some {I', bo) if there is transition with a reset 
of clock X. Assume that this value is "e /c" and consider a point {l,v) with v e 
Then any duration in (0; — v) is sufficient to reach $2 in more than k time units in case 
of an optimal strategy: note that this fact does not depend on e. Given a value "> k" for 
the rightmost position of we can deduce the function a for any position v in the 

interval: it is ftj+i — v + k. 

Therefore (1) the optimal strategies use only the singular points and the rightmost 
positions in the intervals, and (2) the function a over an interval can be derived from 
the value in the rightmost position. Thus we will restrict the computation of coefficients a 
to these points. 

Thus the algorithm consists in computing the function a by using values of the form 
"< fc" , "= /c" or "> k" . This is slightly more technical than the basic case. 

Finally similar techniques can be used also for the other functions (/?, 7 and 5). 
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